Assessing the Reliability of Student Evaluations of Teaching (sets) with Generalizability Theory

This paper presents a Generalizability Theory (GT) analysis of the dependability of measures obtained using the Seevers, Dormody and VanLeeuwen (1998) SET instrument. The analysis of Seevers , et al. (1998) data suggests that both student means and class means are sufficiently reliable measuresfor most purposes. These data are used to illustrate the GT estimation of reliabilities for both student means, involving averaging over items, and class means, involving averaging over both items and students. The use of GT to obtain standard errors for use in interpreting differences in class means is also illustrated and recommendations on the interpretation of class means are given.